Authoring system and method for supplying tagged media content to portable devices receiving from plural disparate sources

ABSTRACT

A system and method for supplying media content from disparate sources includes tagging media content with an identifier tag useful in synchronization with additional media content, and useful in delivery of the media content. In other aspects, the media content is transmitted to a distribution mechanism adapted to distribute the media content to media delivery devices. In yet another aspect, the media delivery device is adapted to obtain additional media content from a disparate source, adapted to synchronize the media content with the additional media content according to the identifier tag, and adapted to deliver the media content to a consumer according to the identifier tag.

FIELD OF THE INVENTION

The present invention relates generally to interactive television andmultimedia systems. More particularly, the invention relates to anauthoring system and method for supplying tagged content that will allowa handheld device in communication with a broadcast source and otherinformation sources to synchronize the information received.

Interactive television holds much promise as the information deliverysystem of the future. It also presents a number of problems that remainto be solved. There are various methods for deploying a multimediaexperience. One method focuses on using a broadcast medium, such as thebroadcast television medium, as the basic information channel, to whichinteractive capabilities will be added. Another method advocates the useof packet-based (e.g., Internet) media delivery channels that wouldsupport both streaming video and user interactive features. Yet anothermethod utilizes external memory sources such as compact disk (CD),secure digital (SD), etc. through suitable distribution channels.

We believe that these interactive multimedia delivery approaches can beenhanced through integration with a variety of handheld portable devicesthat are capable of receiving information from a plurality of disparateinformation sources. By way of example, audio/video remote controldevices may be enhanced to include the capability of wirelesscommunication with broadcast delivery sources, packet delivery sources,and memory-based sources. So equipped, the handheld remote could serveas a very useful component of an interactive television for interactivemultimedia system. The remote control would be capable of receivinginformation from the broadcast source, including electronic programguide (EPG) information and other digital content. The remote controlwould also be capable of receiving digital information from otherinformation sources such as the internet, by wirelessly communicatingwith a suitable portal device such as a gateway. The remote control isalso capable of accessing digital information from other memory basedstorage sources such as CD, SD, etc. The information received from thesedisparate information sources would then be integrated or synchronizedwithin the remote control device to support a variety of differentinteractive features.

Because the remote control device is typically used within thetelevision or multimedia viewing environment, it serves as an excellenthandheld device for providing enhanced interactive experiences. Theremote control device is not the only portable device with which theinvention may be used, however. As will be appreciated from thefollowing specification, the principles of the invention can be usedwith a variety of different portable devices. In this regard, there arecurrently numerous portable devices in popular use. These includepersonal digital assistants (PDAs), cellular telephones, audio/videoremote controls, portable radios, televisions and video players, and thelike. As these devices become more powerful, many expect the functionsof these devices will merge. This merger will transform the portabledevice into a powerful platform for delivery of commercial services toconsumers.

A promising application for portable devices is to integrate mediacontent received from disparate sources, such as from broadcast mediaand from the internet. One problem that system designers currently facein this regard is how to implement the backchannel (the channel by whichthe consumer communicates back to the content provider). Providing abackchannel is comparatively easy on the internet, but it is moredifficult to do so with broadcast sources, because broadcast ischaracteristically one-way communication, hence alternativecommunication methods must be deployed.

SUMMARY OF THE INVENTION

The present invention address the foregoing problems by providing anauthoring system and method that allows content suppliers to tag datathat will later be delivered through different disparate sources. Bysupplying tags during the authorship phase, the tagged data can beautomatically integrated later within the user's portable device—even ifthat portable device is not able to establish a backchannel connection.

The invention includes a high level data flow system and method wherebythe mobile device receives information from plural disparate sources,including at least one broadcast source. An authoring system tags theinformation being sent to the mobile device and the mobile deviceincludes synchronization systems that use the tags to integrate thedisparate information for use in a handheld device. While it is possibleto include a backchannel in the handheld device, a backchannel is notrequired as the authoring system pre-tags information to allow themobile device to integrate it. In one embodiment the broadcast sourcecomprises an analog video source, such as a conventional NTSC, PAL, orSECAM broadcast source. In another embodiment the broadcast sourcecomprises a digital video source such as an MPEG broadcast source. Andyet another embodiment the broadcast source comprises a streaming mediasource delivered using a packet-based delivery system over a networksuch as the internet.

As will become more apparent from the following detailed description,the invention makes possible an information delivery architecture thatwill work regardless of whether a backchannel is ordinarily provided.The invention allows broadcast system operators and media developers tointegrate content that can then be delivered over multiple, diversesources. Thus broadcast media and internet information can be integratedand used by the end user operating one of a variety of differentportable devices. The invention thus enhances not only the broadcastsystem platform but also the portable device platforms (e.g., cellulartelephones, audio/video remote controls, personal digital assistants andother smart consumer appliances).

Further areas of applicability of the present invention will becomeapparent from the detailed description provided hereinafter. It shouldbe understood that the detailed description and specific examples, whileindicating the preferred embodiment of the invention, are intended forpurposes of illustration only and are not intended to limit the scope ofthe invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will become more fully understood from thedetailed description and the accompanying drawings, wherein:

FIG. 1 is a block diagram of an exemplary multimedia system illustratingthe use of the authoring system in an exemplary application;

FIG. 2 is a detailed block diagram illustrating one presently preferredsystem architecture for delivery of tagged content to a portable devicesuch as a PDA, and showing the data flow whereby the authoring systemprovides tagged content;

FIG. 3 is a data flow diagram illustrating how the authoring systemprovides tagged content and further illustrating how the portable deviceintegrates that tagged content;

FIG. 4 is a block diagram of one a presently preferred developmentsystem utilizing the authoring system in accordance with the invention;

FIG. 5 is a data flow diagram illustrating how the authoring systemembeds tags in the broadcast and web page media;

FIG. 6 is a functional block diagram of the authoring system.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The following description of the preferred embodiment(s) is merelyexemplary in nature and is in no way intended to limit the invention,its application, or uses.

Although the invention can be deployed in a wide variety of differentapplications, FIG. 1 presents an exemplary application that focuses onthe interactive television environment. Referring to FIG. 1, thetelevision 10 and the associated set top box 12 receive a modulatedsignal from the broadcast source 14. Typically, the signal is modulatedonto different channels 30 that are tuned preferably by the set top box12. Alternatively, a digital content decoder can also be built into TV,or the portable device, or use a separate appliance box that is pluggedinto TV externally. As will be more fully explained below, the set topbox not only selectively tunes to the channel of the user's choice, butit also decodes digital content that is placed in the broadcast screenby the content provider using the authoring system 16 of the invention.This digital content is communicated, preferably wirelessly, to thehandheld device 18 (in this case a remote control device with PDA-likeviewing and data entry screen 20).

The authoring system 16 also delivers digital content to a suitableserver 22 that is coupled to an information delivery network such as theinternet 24 allowing it to act as a portal through which users canaccess the digital content using computers and other devices. A majorfunction of the authoring system is to send the interactive data tovarious sources (e.g., broadcast sources, internet, cellular, portablememory devices, and the like) from a single point of content creation.This is illustrated in FIG. 6.

Referring back to FIG. 1, the authoring system can also deliver digitalcontent for delivery through a variety of different media deliverychannels, including broadcast sources, the internet, and throughalternate media delivery mechanism 23. Such mechanisms includememory-based storage systems, CD-ROMs, flash memory, SD media and thelike. In the illustrated embodiment the user has a home gateway device26 communicating via internet 24. The home gateway thus suppliesinternet conductivity to personal computers such as computer 28 and alsoto the handheld device 18, preferably via a wireless connection. Thusthe handheld device 18 receives digital content from the internet 24 aswell as from broadcast source 14.

In the embodiment illustrated in FIG. 1 the set top box 12 and homegateway 26 have been shown as separate units. In practice, the two maybe included in a single unit that would supply both broadcast sourceaccess (e.g., cable television or satellite channels) as well ashigh-speed internet access (e.g., broadband access). Communication withthe handheld device can be through a variety of different wirelesstechnologies including infared technology, spread spectrum wirelesstechnology Bluetooth technology, IEEE 802.11, and the like. This samewireless communication capability can be used to permit the handhelddevice to wirelessly communicate with other devices such asdiagrammatically illustrated at 30. These other devices include adiverse list ranging from point-of-sale (POS) terminals to globalpositioning systems, to car navigation systems, VCR and DVD recordersand to other smart appliances. In this way, the handheld device might beconfigured to receive information from broadcast source 14, from portal22 and then integrate that information for use in accessing orcontrolling some other device 30. Alternatively, the other device may becontrolled by supplying it with data from some other media, such asmedia 23 a that may be furnished by the other media distributionmechanism 23.

Continuing with the interactive television example introduced in FIG. 1,FIG. 2 shows in greater detail how the authoring system 16 may supplytagged content through both analog and digital video broadcast sourcesand through packet-delivery sources such as an internet portal source.Referring to FIG. 2, program content that will ultimately be deliveredto the viewers television 10 has been illustrated diagrammatically at50. This program content can include both entertainment content andadvertising content, as desired. To provide an interactive experience,certain digital content will be associated with the program content.This digital content is diagrammatically depicted as data elements 52,shown in various locations throughout FIG. 2. In the case of an analogbroadcast system, the program content 50 is supplied as analog videocontent 60 to which the digital content 62 (derived from data 52) isadded through an encoder device 64. In this regard, the current verticalblanking interval (VBI) encoder defines protocols by which digital dataare embedded in “invisible” lines in NABTS packets in an NTSC system.Once the analog and digital components have been combined in theencoder, the signal is sent to modulator 66 for placement onto thebroadband delivery system such as cable system 68.

In a digital video implementation, both program content 50 and digitaldata content 52 may be combined directly as digital video content, asillustrated at 70. In this regard, the current MPEG-2 standard definesprotocols by which both program content and data content can be combinedinto transport stream data packets for delivery to the customer. Thedigital video content is thus modulated in modulator 72 and then addedto the broadband delivery source 68.

FIG. 2 has been subdivided by a dashed line into the upper half, whichrepresents the content creation portion of the system and the lower halfwhich represents the receiving end of the system. With reference to thereceiving end portion of FIG. 2, the broadband delivery system 68preferably couples to the set top box 12. In FIG. 2, the set top boxdecoder and tuner functions have been separately labeled. The tunerfunctionality selects one of the channels for displaying program contenton the television screen 10. If any digital content has been encoded onthis channel, the decoder functionality within set top box 12 extractsthat digital content so that it may be supplied to the handheld device18.

With regard to the digital data and the decoder functionality, theinvention is capable of supporting a variety of different dataprotocols. When a digital video signal is provided, the digital contentcan be extracted from the data stream by using a transport streamdecoder. When analog video content is provided, the digital content maybe encoded within the vertical blanking interval (VBI). In FIG. 2, it isassumed that the user has tuned to channel 3 (the analog channel) thus'the digital content is encoded as VBI data. This VBI data is initiallystored preferably at 80 within the memory of the set top box 12. Inaddition to the memory 80, the set top box includes a tuner and thenecessary decoder functionality to extract the digital content (in thiscase illustrated as VBI content). These functions have been illustratedin FIG. 2 as functional components within set top box 12.

The set top box also preferably includes wireless communicationcapability to allow it to communicate with the portable device 18. Usingthis communication capability, the contents of memory 80 are transferredto the memory 84 within the portable device 18. The portable device 18includes system software that is capable of parsing and using thistransferred data to support a number of applications running on thehandheld device, as will be more fully explained below.

While a handheld device such as device 18 represents a presentlypreferred embodiment, the decoder and tuner may also be configured tocommunicate with a personal computer and thereby supply VBI data (orother forms of encoded digital data) to the computer as well.

Once the digital data is stored within the handheld device, it maypropagate by wireless communication or hard-wired communication to otherdevices such as other PDA devices, personal computers 28, cellulartelephones, and the like.

In the illustrated embodiment, the transferred data stored at 84 mayactually represent electronic program guide (EPG) information. Thus,FIG. 2 depicts the digital content at 84 as EPG information. Thehandheld unit 18 may be docked to a personal computer, such as computer28 for configuration, if desired. The handheld unit 18 alsocommunicates, preferably wirelessly, with an internet access point, suchas home gateway 26, which in turn communicates with the internet 24. Inthis way, the handheld device 18 is able to receive information from theportal server 22 (shown in the upper half of FIG. 2). While EPGinformation has been illustrated at 84 it will be appreciated that theinvention can receive any type of digital information from the broadcastsource. EPG information is particularly useful in interactive televisionapplications, because it can supply information about the program towhich the user is tuned. This information is then integrated with otherinformation about the program being watched that may be received via theinternet through portal 22 or other external memory sources. The tagssupplied by authoring system 16 provide the synchronizing capability aswill be more fully explained below.

FIG. 3 shows an alternate architecture of the set top box 12 andportable device 18 in somewhat greater detail. Specifically, FIG. 3shows how an exemplary system might be constructed to utilize VBI data.A similar architecture may be used to support other types of data (i.e.,data that is not encoded for delivery within the vertical blanketinterval, but rather within the main digital video transport stream).

Referring to FIG. 3, the incoming TV signal is illustrated at 100. Itpasses through an optional splitter 102 which feeds the signal to thetelevision 10 and also to the decoder components of the set top box 12.Specifically, the incoming signal is supplied to the VBI decoder 104.the VBI decoder is preferably implemented as a hardware device thatextracts VBI data from the incoming TV signal. The decoder can specifythe TV channel and decode the VBI data transmitted in that channel.Typical VBI decoder cards available today provide on the order of 230kbits per second.

The decoder 104 supplies an output signal to the VBI data extractor 106.Because VBI data are sent only at certain time intervals on eachchannel, the VBI data may not be available at the instant the userrequests access to it. The VBI grabber or extractor 106 is a softwaremodule that constantly grabs or extracts all available VBI data andstores it in VBI data buffer 108. The data grabber channel stamps andtime stamps all VBI data so that the system will later be able todetermine any necessary temporal or channel information needed to usethe data.

By operating the handheld device the user may from time-to-time requestVBI data from the set top box 12. The user command processor 110 withinthe set top box 12 responds to these requests and accesses the VBI databuffer 108 to obtain the information the user has requested. The commandprocessor has an infared transmitter module 112 that allows it tooperate as a TV remote control signal source. The processor is alsocoupled to a wireless interface device 114 that allows it to communicatewirelessly with the handheld device. In the presently preferredembodiment both IEEE 102.11B or Bluetooth interfaces are presentlypreferred.

The handheld device includes similar wireless communication capabilityin the form of wireless communication module 116. This module suppliesdata to a VBI database 118. The database is accessed by a VBI dataparser 120 which in turn supplies information to the data requestprocessor 122. The processor 122 supplies selected VBI data to anynumber of application programs 124 that are operating within thehandheld device. If desired, handwriting analysis module 126 can be usedto interpret a user's information requests and operating instructionsthat have been entered by a stylus. These handwriting instructions arethen decoded by the module and used as instructions by the module'sintegrated search engine. The search engine supplies search instructionsto the data request processor 122, as illustrated.

The handheld device also includes a synchronization engine 128 thatpreferably communicates directly with the VBI database 118. Thesynchronization engine performs the integration function by which VBIdata and digital data downloaded from the internet are synchronized foruse by the application program. The synchronization engine 128 makes useof the data tags that are applied by the authoring system 16 (FIG. 2).

The authoring system by which tags are added to the media content and tothe other content such as internet web content can be implemented in avariety of different architectures. A presently preferred architectureis illustrated in FIG. 4. The authoring system 16 may be configured asan application server that integrates with a plurality of differenteditors, each associated with a different type of media delivery system.In the illustrated embodiment the authoring system communicates with avideo media editor 140, a web development editor 142 and an other systemdevelopment editor 144. This other system development editor isillustrated in FIG. 4 to demonstrate that the authoring system iscapable of communicating with a wide variety of different types of mediaeditors. The other system might be, for example, a wireless system thatprovides content to portable devices such as cellular telephones andsuitably equipped personal digital assistants.

Each of the editors is responsible for manipulating its associated typeof media content and ultimately delivering that content to a suitablemedia server for ultimate delivery to the viewers. In FIG. 4 the videomedia editor supplies its content to a media server 146 that might benet coupled to a broadcast network (e.g., television broadcast, cable TVhead end or satellite system). The video media editor may have anassociated database 150 containing program content being manipulated bythe editor. As illustrated, the authoring system 16 may, also be coupledto one or more databases containing program content, as in database 152and other interactive content as in database 154. The authoring systemwould respond to requests from the editor (e.g., video media editor 140)to integrate program content and other interactive content into thefinal product.

In a similar fashion the web development editor 142 supplies its outputto a web portal 22. The other system development editor likewisesupplies its output to a suitable portal 156. In the illustratedembodiment the portal 156 has been designated as a wireless portal asmight be used to supply content to portable devices using cellular orother wireless technology.

In addition to supplying program content and other interactive contentto the various editors, the authoring system 16 serves the importantfunction of generating and supplying to each of the editors a suitabletag that may be integrated into the media being edited by each of therespective editors. These tags are each designed to be compatible withthe media format being developed by each editor. Although the format ofthe tags will depend on the nature of the media, the content of each tagis the same in this respect. The content stores synchronizationinformation that the viewers or end users handheld devices, audio videoequipment and other intelligent appliances used to synchronize theinteractive components of media received from various disparate sources.The concept is illustrated in FIG. 5.

Referring to FIG. 5, the authoring system 16 delivers tags 200 to theeditors for insertion into the media being edited. In the illustratedembodiment shown in FIG. 5, the media content being edited by videomedia editor 140 is shown at 202 as a sequence of video frames separatedby a vertical blanking interval 204. As illustrated, tags 200 areinserted into the vertical blanking intervals using a format that iscompatible with the video data stream. In digital broadcasting, the tags200 are inserted into data packets that are compatible with thetransport stream.

FIG. 5 also illustrates at 206 the media content being edited by the webdevelopment editor 142. As illustrated, the authoring system suppliestags 200 that are inserted into the supplemental content. Specifically,the supplemental content illustrated at 206 is in the form of a web pagebeing developed using HTML and optionally other web developmenttechnologies. The tag 200 appears in the HTML source code as a SYNCTAGof a specified type, in this case of the “interactive” type. Embeddedwithin the tag a plurality of parameters (Parameter A, Parameter B . . .). These parameters represent synchronization information that are usedto integrate the supplemental content with the associated video contentcontaining corresponding tags as illustrated at 202.

In the presently preferred embodiment the tags 200 are effectivelyhidden within the data streams of the respective media servers. Viewerswho do not own suitably equipped devices are able to view the contentdelivered by the respective media servers without any awareness that themedia have been tagged. Viewers who are using suitably equipped devices,such as the handheld device illustrated in FIGS. 1, 2 and 3, are able toenjoy the interactive experience made possible by these embedded tags.The device parses the data stream, extracts the embedded tags and theninterprets them to provide the application software running on thedevice with the necessary synchronization information.

The description of the invention is merely exemplary in nature and,thus, variations that do not depart from the gist of the invention areintended to be within the scope of the invention. Such variations arenot to be regarded as a departure from the spirit and scope of theinvention.

1. A media distribution system adapted to supply media content from disparate sources, comprising: an encoder tagging media content with an identifier tag useful in synchronization with additional media content, and useful in delivery of the media content; an output transmitting the media content to a distribution mechanism adapted to distribute the media content to media delivery devices; and a distribution mechanism distributing the media content to a media delivery device adapted to record a channel upon which the identifier tag arrived, adapted to obtain additional media content from a disparate source, adapted to synchronize the media content with the additional media content according to the identifier tag and the channel, and adapted to deliver the additional media content to a consumer according to the identifier tag and a remote channel control function of the media delivery device.
 2. The system of claim 1, wherein said distribution mechanism is adapted to broadcast the media content to media delivery devices.
 3. The system of claim 1, wherein said distribution mechanism is adapted to narrowcast the media content to a media delivery device in response to a request for the media content.
 4. The system of claim 1, wherein said encoder is adapted to insert an identifier tag into a vertical blanking interval of a sequence of video frames using a format that is compatible with a video data stream.
 5. The system of claim 1, wherein said wherein said encoder is adapted to insert a tag into a web page.
 6. The system of claim 1, wherein the media content corresponds to a textual description.
 7. The system of claim 1, wherein the media content corresponds to an image.
 8. The system of claim 1, wherein the media content corresponds information in a rich text format.
 9. The system of claim 1, wherein the media content corresponds to information in a binary language format.
 10. The system of claim 1, wherein the media content corresponds to a link to the additional media content via a communications system.
 11. The system of claim 1, wherein the media content corresponds to audio visual media content.
 12. The system of claim 1, wherein said encoder is adapted to select the identifier tag based on material characteristics of the media content.
 13. The system of claim 1, wherein said encoder is adapted to insert the identifier tag into a real-time distribution channel.
 14. The system of claim 13, wherein said distribution mechanism is adapted to employ a real-time distribution channel to ensure delivery of the media content at a scheduled time without being affected by a distribution load.
 15. The system of claim 1, wherein said encoder is adapted effectively to hide the identifier tag within a data stream to ensure that consumers not having suitably equipped media delivery devices are able to consume the media content without any awareness that the media content has been tagged, and to ensure that consumers having a suitably equipped media delivery device are able to enjoy an interactive experience facilitated by the identifier tag.
 16. A method for supplying media content from disparate sources, comprising: tagging media content with an identifier tag useful in synchronization with additional media content, and useful in delivery of the media content; transmitting the media content to a distribution mechanism adapted to distribute the media content to media delivery devices; and distributing the media content to a media delivery device adapted to obtain additional media content from a disparate source, adapted to synchronize the media content with the additional media content according to the identifier tag, and adapted to deliver the media content to a consumer according to the identifier tag.
 17. The method of claim 16, wherein said distributing includes broadcasting the media content to media delivery devices.
 18. The method of claim 16, wherein said distributing includes narrowcasting the media content to a media delivery device in response to a request for the media content.
 19. The method of claim 16, wherein said tagging includes inserting an identifier tag into a vertical blanking interval of a sequence of video frames using a format that is compatible with a video data stream.
 20. The method of claim 16, wherein said tagging includes inserting a tag into a web page.
 21. The method of claim 16, wherein the media content corresponds to a textual description.
 22. The method of claim 16, wherein the media content corresponds to an image.
 23. The method of claim 16, wherein the media content corresponds information in a rich text format.
 24. The method of claim 16, wherein the media content corresponds to information in a binary language format.
 25. The method of claim 16, wherein the media content corresponds to a link to the additional media content via a communications system.
 26. The method of claim 16, wherein the media content corresponds to audio visual media content.
 27. The method of claim 16, comprising selecting the identifier tag based on material characteristics of the media content.
 28. The method of claim 16, wherein said tagging includes inserting the identifier tag into a real-time distribution channel.
 29. The method of claim 16, wherein said transmitting and distributing include employing a real-time distribution channel to ensure delivery of the media content at a scheduled time without being affected by a distribution load.
 30. The method of claim 16, wherein said tagging includes effectively hiding the identifier tag within a data stream to ensure that consumers not having suitably equipped media delivery devices are able to consume the media content without any awareness that the media content has been tagged, and to ensure that consumers having a suitably equipped media delivery device are able to enjoy an interactive experience facilitated by the identifier tag.
 31. A method of disseminating information for use in a portable device, comprising: generating first content adapted for dissemination from a broadcast source; generating second content adapted for dissemination from an information source; using an authoring system to apply tags to said first and second content, the tags being configured to define an integrating relationship between the first content and the second content; providing said first content to a broadcast source adapted to broadcast to a portable device; providing said second content to an information source adapted to supply information upon demand to said portable device; and providing the portable device, wherein the portable device has ability to synchronize and deliver said first content and said second content based on a channel upon which said first content was broadcast to the portable device, and based on a channel remote control function of the portable device. 